since it looks like you'll be changing things for a bit, i'll hold off until you tell me you're at a good endpoint or intermediate point. no sense overdoing it.

Sounds good, I'll mark this as a draft until I reckon you should merge it

The problem with Debian and Ubuntu (and likely the other distros, although they have other outstanding issues) is the deprecation of webkitgtk-3.0 (see #794) which is also preventing me from building on my Debian system.

The only reason the package in in the Debian repos is a patch from the maintainer removing the webkit editor (see here).

This seems to be a long standing issue with a lot of work needed to fix but at least it explains the failure of the CI builds.

just curious, can you tell me where you are with this effort?
i'm trying to get a few things accomplished (notably several PRs completed) and this remains in draft state.
if it needs to wait, that's fine, i just wonder where you are, how far you've gotten.

Unfortunately I'm unable to make progress until we decide on a route to solve the webkitgtk debacle, since the reason the CI builds are failing is due to missing dependencies on said webkitgtk libraries.

I mentioned this at the start of the email I sent on the xiphos-devel mailing list, so you can refer to it for a more in depth explanation.

If you're happy with taking the route that Debian has (i.e. removing the components, notably the studypad, that require the webkitgtk dependencies) I'd be happy to do so, but it would be a large set of features to remove.

i did respond to your email to xiphos-devel, and offered some explanation for what i thought i knew. perhaps i was off, but my belief was that webkitgtk was simply moving forward to a most-recent 4.1 version, hence recent commits to xiphos/cmake/XiphosDependencies.cmake, still distinguishable from the old versions that the windows build still needs.

looking at the latest build logs, it seems the fedora and arch failures are strictly on the version stamp:

CMake Error at CMakeLists.txt:73 (project):
  VERSION ".128" format invalid.

looking at the debian failure, it's unhappy with libenchant-dev; is this now missing, or renamed, or...? ubuntu is complaining about python-dev -- is this needing a version spec, such as perhaps python3-dev (i'm only a very casual ubuntu user)?

sorry if i'm being dense, but i don't see precisely what's gone wrong in this instant, specifically how it relates just to webkitgtk.

if the editor is the real base of the troubles, then #ifdef'ing it out (viz studypad and other editables) as a build-time choice would be ok, and let those features continue where the platform allows (fedora, at least).

Sorry for the late reply, was away this week just past.

Regarding the Arch and Fedora builds you are completely right that the version number is breaking them. I hadn't focused on them too much, apologies. Unfortunately I have no clue how to solve that, I'm not that familiar with CMake so if anyone has expertise there help would be greatly appreciated.

Debian and Ubuntu are both missing libenchant-dev which is required to do the manual build and install of gtkhtml. So we need some replacement. I tried upgrading it to libenchant2 but that requires changes to the underlying code of the gtkhtml source that is grabbed from the archives (which is a massive hassle and leaves a bad taste in the mouth for any distro packagers).

As for Ubuntu specifically I'll change the CI to use that updated webkit2gtk 4.1 version. The python-dev issue I thought I had fixed, that's a really simple change just need to rename to python-dev-is-python3, will check if there's anywhere I forgot to rename the package. EDIT: I had fixed it, the changes don't reflect on the master branch's CI builds but if you look at the pull request's CI builds the python-dev error is not present.

As for the #ifdefs I'll go ahead with that. You say Fedora wouldn't need it, does Fedora have the old gtkhtml packages in its repos still?

So as for the relation to webkitgtk, it seems to be primarily gtkhtml instead. In all honesty I am still unsure of what the difference in uses is for these two libraries, do they not serve the same purpose? If they do, I would propose removing gtkhtml entirely. It also seems those earlier commits you were referring to didn't change all the references to the new webkit2gtk version, so I'll go about changing the rest of them.

Thanks for your help and support, feel free to correct me if there's anything further you see needs altered.

fedora continues to have gtkhtml. xiphos may be the last project using it. i think arch also still has it.

we don't use a lot of gtkhtml; it's used only for the editor (libgtkhtml-editor.so), and has for many years been a serious pain in our collective backside. it is desperately in need of being re-written against some other toolkit besides gtkhtml, but no one, including me, has been willing to wade into the swamp. :-/

the version number confusion that causes fedora/arch build failuire has me stymied. i asked several folks privately a couple months ago about it, and sadly no one seems to remember how the overall mechanism works. i didn't implement it; it was part of the cmake conversion when we left waf behind, which was handled by others.

your contributions and assistance are hugely appreciated. sorry that interaction here has been slow on both our parts.

A few notes about the CMake snippet (CMakelists.txt: lines 63-67)

The code first checks if a .git directory exists in the project source directory:

if(EXISTS "${PROJECT_SOURCE_DIR}/.git")

If the .git directory exists, it includes a CMake module called GetGitRevisionDescription:

include(GetGitRevisionDescription)

It then uses the git_describe function (provided by the included module) to get the Git version information:

git_describe(GIT_VERSION --tags)

The code applies a regular expression to modify the version string:

string(REGEX REPLACE "-([0-9]*).*" ".\\1" VERSION "${GIT_VERSION}")

This regex replaces "-" with ".". For example, "v4.2-3-g123abc" would become "v4.2.3".
Finally, it writes the processed version string to the file cmake/source_version.txt:

file(WRITE ${SOURCE_VERSION_CACHE} "${VERSION}")

$(VERSION) format:
<major>.<minor>.<patch>[-rc<n>]
where the component is less than 20000000.

To sum up, this CMake code automatically generates a version information based on Git tags. It allows the build system to incorporate the current Git version into the project's version number, which can be useful for tracking builds.
The release tarball doesn't include a .git directory, the version information is then based on the content of /cmake/source_version.txt.

@domcorbex
where/how does the fedora failure arise?

CMake Error at CMakeLists.txt:73 (project):
  VERSION ".128" format invalid.

there should be no .git directory in (what i presume must be) the generated tarball from which to work. how does it find ".128" -- from where does this come? (cmake/source_version.txt is not in the tree, and is named in .gitignore.)

IMHO, the script:

1. Checks if the project contains a Git repository.
2. If it is, it reads the version from git tags, processes it, and saves it in SOURCE_VERSION_CACHE.
3. If it's not, it reads the version from SOURCE_VERSION_CACHE

In the case of a tarball, the .git folder doesn't exist, CMake reads the version from SOURCE_VERSION_CACHE (cmake/source_version.txt).

If I'm not mistaken, the problem is that SOURCE_VERSION_CACHE is only created in the tarball by cmake package_source. In the CI, I guess the repo is exported without the .git directory, so both the .git directory and SOURCE_VERSION_CACHE (cmake/source_version.txt) are missing, as a result VERSION is undetermined.

A quick fix would be to create cmake/source_version.txt and push the next version string in it.

I don't know what's going on with CI, the full repo seems to be fetched, with tags and so on.. (.github/workflows/builds.yml)

The latest git cloned on Fedora 40 builds successfully, VERSION=4.2.1.27

Thanks very much for the guidance @domcorbex ! That's the CMake version stuff sorted out temporarily (currently just sets the version to 0.0.1). We've got some new errors from Arch and Fedora now though.

Arch can't find glib-genmarshal. No clue what this is or what might be a replacement package.

Fedora can't find libsoup-3.0, which seems to be called libsoup3 on Fedora's repos, but I don't know how to tell CMake to differentiate between distros. I'll look into this one.

I have no clue how the Windows CI builds work, it looks like @greg-hellings created it, so if you are reading this Greg any help would be appreciated!

All the Linux builds are working. If I remove the studypad button from the top bar menu, @karlkleinpaste , would you be happy to merge?

just to clarify, for those platforms which still provide gtkhtml4-editor, is the editor itself still enabled by this? i was looking through the commits, and the changes in webkit-editor.c give me pause, though i haven't really looked closely at the details yet.

in general, i'm totally ok with merging soon; if you have something remaining to be done, well, just let us know.

@greg-hellings could we convince you to invest what i hope would be no more than an hour or two to re-solidify the windows build mechanisms, including (i suppose) a needed update to the sword patch?

Ah my bad, I had removed the editor from them all to keep it consistent, I'll re-enable it for those distros that do provide the necessary packages

on the matter of the version stamp causing build failure...

i have a xiphos tree that was built yesterday sometime. i realized i hadn't installed on the other near-identical machines nearby. so i went into the 1st of these, with this existing, successful build in place, and did make install. it failed:

# make install
CMake Error at CMakeLists.txt:73 (project):
  VERSION ".128" format invalid.
-- Configuring incomplete, errors occurred!
make: *** [Makefile:1133: cmake_check_build_system] Error 1

@greg-hellings can you offer insight into how this occurs, what it means? we see this routinely, every time a commit is made and auto-build is attempted, and workarounds so far here on @Phyrik 's branch have been simply to obliterate the logic entirely for the time being. the error occurs on master branch.

from where does ".128" come? where is it found? this simple pipeline finds nothing:
fgrep -r .128 2>&1 | fgrep -v .po | grep -v 'binary file'

if i re-run the entire build from scratch, then this tree installs on all systems -- because it's then "fresh"?

Part update report and part status update request...

As you've probably seen, I just made a smallish code change to handle differences required by RHEL9's gcc11. They're small, they didn't hurt recent Fedora, and I hope they don't break anything else. Please feel free to examine, apply, and test on whatever other systems you've got.

Beyond that, I'm wondering where this PR's effort is now. I know I've been struggling toward release for far too long, but I'm hoping we're getting close. I think what's needed is to get back to retaining the editor for those platforms still supporting libgtkhtml-editor and resurrecting the Windows side of the build.

I didn't have to do anything for Sword or BibleSync in RHEL9.

The configuration oddities of RHEL9, requiring libsoup2 when the world has otherwise moved forward to libsoup3, and needing webkit2gtk 4.0 instead of 4.1, represent a problem for cmake/XiphosDependencies.cmake. The use of pkg_check_modules needs to become more sensitive, and I'm not sure how to do this. Would it suffice to add new cmake -D options to specify a platform directly (i.e. -DRHEL9) so that this can then be tested individually? Can this be done overall for other platforms as well, notably Windows' similar need for libsoup2 and older webkit2gtk? I'm not fully comfortable with recommended cmake-isms and hope @greg-hellings in particular can advise.

@Phyrik @greg-hellings
I can hardly be one to push too hard, but I am wondering if there is any update.

Sorry for the long hiatus, started university recently so was a bit swamped. Hoping to get stuck back in to this now though.

Regarding this PR in particular my outstanding issue is with the Windows build on Fedora. I have no clue how to find the mingw versions of the relevant packages to fulfill the build's dependencies. Will have another try some point soon but might need your help @greg-hellings if you are willing to offer it. It has been much appreciated thus far.

I will say that, as a general rule, it is considered a poor choice to hard-code a specific thing like one distribution. Unless it is an environment where you control the whole build to binary process and release - such as we do with the Windows release. Doing it for a RHEL-derived distribution would be considered an anti-pattern. Better to discover features than distro names and versions. However, if you are comfortable with maintaining those specific settings for each target environment, then the main reason it is an anti-pattern is the potential maintenance that comes with it. Eventually you end up maintaining a different definition file for each distribution, OS, etc. So deciding that option is really a maintenance decision - are you more comfortable with some more complicated but generic detection algorithms or are you more happy with a simpler mechanism that might require more manual updating each time a new distro version comes out with new versions of libraries, library renames, etc?

So, while best practices might suggest away from having something like -DRHEL9, it might be worthwhile for Xiphos to break "best practices" in the interest of maintainability.

As for Windows specific build issues, I think the build is pinned to Fedora 30. It's a very old, now, version of Fedora that last had all the dependent libraries we require for Windows because of webkitgtk dropping support back in like 2.24 for Windows. If the versions of the libraries we need are not available on that distribution, we will need to figure out a way to build them for it. I have, since, learned better ways to pin builds, but for now let's focus on those antique versions of libraries available in Fedora 30 before we try to tackle Yet Another Build Mechanism.

I'll look into what the build failures are here today and post back with what I find.

OK, so a little bit of digging here:
webkit2gtk4.0 is for GTK3 and libsoup2
webkit2gtk4.1 is for GTK3 and libsoup3

libsoup3 was released in 2021 and simply won't be available for Fedora 30 (our base for Windows builds), which went end of life in 2020. Also, you won't find it supported in the Windows build of WebKitGTK which was last supported in - I believe - 2.24 released in 2019.

If you want to keep support for Windows builds you will either need to back out support for libsoup3 or maintain support for both libsoup2 and libsoup3 in the builds with the Windows version leveraging the libsoup2 and the old webkit1 support for as long as that is the display kit in use.

The easiest path going forward might be to use the official port of WebKit for Windows, which is based off Cairo. There appears to be a pretty solid way to connect GTK and Cairo together, since GTK has used Cairo to render most (since 2.8) or all (since 3.0) off its control widgets. I don't pretend to be an expert in GTK or Cairo, but the web is replete with examples of how to grab a Cairo pointer that can be used to render onto a widget.

But, for the purpose of this PR, you'll need to find a way to leverage the libsoup2/webkit2gtk4.0 API and libraries for the Windows build.

I just wanted to mention -- a couple days late, sorry -- that I'm not ignoring this, but I'm in south Florida and the last week has been kinda overly exciting around here. I'll respond some tomorrow, probably.

I think the summary is that the 4 options, in descending order of effort are:

1. Change the display technology used on Windows away from WebKitGTK - an option that has been hashed over many times
2. Create packages for libsoup3/webkit2gtk-4.1 with the MinGW toolchain for Fedora 30
3. Move away from using Fedora 30 for the builds. I've just recently been experimenting with building for Windows using the Nix tooling. This doesn't get away from the fact that WebKitGTK does not now and will not in the future support Windows builds and we will still need to use old versions of libraries.
4. Maintain both the libsoup2/webkit2gtk-4 (Windows) and libsoup3/webkit2gtk-next (Linux) source paths in the tree.

Obviously option 4 is the way we have handled this in the past for things like the editor and maintaining GTK 2 for compatibility with WebKitGTK on Windows. But it throws the effort mostly back into the coder's face to work around the two code paths.